Assembling Furniture by Asking for Help from a Human Partner

Robots inevitably fail, often without the ability to recover autonomously. A human partner can help a robot recover from a failure, but it is challenging even for a willing and motivated helper to determine what actions will effectively aid the robot. To address this problem, we demonstrate an approach for enabling a robot to communicate its need for specific help to a human partner using language. Our approach automatically detects failures, then generates targeted spoken-language request such as “Hand me the white table leg.” Once the human partner has repaired the failure condition, the system resumes full autonomy. We present a novel algorithm for generating help requests by emulating the human’s ability to interpret a command, based on the Generalized Grounding Graph framework. We report preliminary results for an early version of our algorithm.